1. Field of Invention
This invention relates to devices for remotely controlling electronic equipment, and more particularly, to a wireless, user-programmable, voice-activated and voice-operated remote control system for controlling appliances.
2. Description of Prior Art
Historically, appliances, for example, electronic appliances, such as, televisions, VCRs, digital satellite systems, audio systems, and related accessories, have been remotely controlled by hand-held transmitters used to generate signals to receivers incorporated into the electronics of the remotely controlled appliances. Signals for such appliances correspond to control commands, such as channel selection/tuning, power on/off, audio volume adjustment, and muting controls, typically generated by the user by depressing buttons on a remote control transmitter keypad. The basic composition and operation of such remote control systems are well known in the art.
For convenience, such remote control transmitters are generally designed to be sufficiently small to be hand-held during operation. There are several disadvantages in using such hand-held transmitters. For example, the small size and mobility often contribute to misplacement or loss of the transmitter. Also, for device operators with restricted physical mobility or sight limitations, hand-held remote controls may not provide sufficient access to the command controls of the remotely controlled appliances. Also, if an operator""s hands are engaged in an activity, an interruption in the activity may be required to operate the hand-held remote control, causing inconvenience to the operator and potentially having an adverse effect on productivity.
Additional problems occur as a consumer acquires various remotely controlled appliances, where each is typically supplied with a separate remote control transmitter. As the number of separate remote control transmitters increases, locating, distinguishing, and locating the appropriate transmitters becomes increasingly difficult. In response to this problem, universal remote control transmitters, pre-programmed by the manufacturers with control commands, typically coded infrared signals, have been developed to operate many different remotely controlled electronic appliances, for example, as disclosed in U.S. Pat. No. 4,774,511. In addition to universal remote control transmitters, learning remote control transmitters have been developed which receive the control command signals, typically infrared codes, from the remote control transmitters provided with remotely controlled appliances and generally store the frequencies and pulses into memory, with the signals becoming associated with buttons located on the keypad of the learning remote control transmitter. After programming is complete, depressing the keypad buttons prompts the learning remote control transmitter to re-transmit the codes stored within its memory. This allows the user to consolidate the control of several remotely controllable appliances into a single hand-held remote control transmitter. An example of such a system is disclosed in U.S. Pat. No. 5,142,398. For additional convenience to the user, the learning capability has been combined with universal remote control transmitters which are pre-programmed by the manufacturers with control commands necessary to operate many different remotely controlled electronic appliances, for example as disclosed in U.S. Pat. No. 5,691,710.
All of the above-mentioned systems require the user to establish physical contact, typically in the form of manually depressing keypad buttons, to transmit a control command to the remotely controlled appliance. As such, the above-mentioned systems due to their small size are often misplaced causing frustration to the user.
As such, voice-operated remote control systems have recently been developed which operate in response to user-spoken commands instead of commands selected by manually depressing keypad buttons. U.S. Pat. Nos. 5,199,080; 5,247,580; 5,267,323 and 5,226,090 disclose various embodiments of voice-operated remote control systems which employ voice control commands instead of control commands entered through buttons on a keypad. Unfortunately, such systems are not truly hands-free, requiring manual intervention by the user during use. In particular, such remote control systems as disclosed in the above-mentioned patents, are all based upon the use of a xe2x80x9ctalk switchxe2x80x9d; which must be manually depressed to enter a voice command when the transmission of a remote control signal is desired.
U.S. Pat. No. 5,226,090 further discloses a voice-operated remote control system which contains a detector for detecting whether a voice command is received or not. A detected signal from the detector is applied to a power supply circuit. When there is no voice command received as detected by the detector, the power supply control circuit cuts off electric energy supplied to a speech recognition circuit. The voice command detector circuit is implemented by a detecting program stored in the ROM in a controller. The detecting program determines whether the talk switch has been depressed. The transmitter operates depending on whether the talk switch has been depressed. If the talk switch has been depressed, the transmitter is enabled to remote control signals. Once the talk switch is released, the transmitter is kept in a low power consumption mode, waiting for voice commands to be applied. As indicated above, the means for generating and transmitting a remote control signal based on the recognized spoken voice command is not hands-free, requiring the manual intervention of pressing a talk switch to accomplish these functions.
Various other systems are known which use speech recognition to control appliances. For example, U.S. Pat. No. 5,852,804 discloses a system for controlling several external appliances connected to a speech recognition system. However, the system disclosed in the ""804 patent requires physical interconnections between the control system and the appliance which makes it difficult for a user to add additional appliances or change controlled appliances.
U.S. Pat. No. 5,878,394 discloses a system which includes connections to a remote control network for transmitting infrared codes and a graphical user interface on a personal computer (PC). In addition to the inconvenience of establishing the remote control network, a user must have access to a PC and desire to make the PC available for such control functions. The system disclosed in the ""394 patent has only limited utility for consumers who have access to a PC or can afford the expense of a PC and thus excludes a significant portion of consumers who may otherwise desire to add speech recognition control to their electronic appliances if an inexpensive alternative is available.
U.S. Pat. No. 5,774,859 discloses a subscriber information system with a speech interface, similar to the system disclosed in U.S. Pat. No. 5,878,394 the system in the ""859 patent is based upon a PC-class processor. Operation of this system is dependent upon receiving information from an information distribution center or head-end installation and therefore lacks the advantages of a stand-alone device. Unfortunately, the remote control is not capable of independently completing the speech recognition process and transmitting infrared signals indicated by such recognition results to controlled appliances. Furthermore, this system also requires a PC-class processor.
Inconvenience is another problem that occurs as a user intends to perform multiple control functions involving one or more controlled appliances and must recall and enter a series of voice commands to a voice-operated remote control system. U.S. Pat. No. 5,852,804 discloses a system, where one speech command may be defined which is capable of manipulation of more than one connected appliance. However, this system requires that each controlled appliance be physically connected to the system and that the controlled appliance be manually manipulated during setup for each function to be executed upon recognition of the speech command. Also, if a single command is to be used to execute a series of controlled appliance functions, its capability is limited unless time delays of variable lengths may be inserted within the series of control functions to accommodate the initialization periods for certain controlled appliances. For example, if a voice command sends signals to a television to power it on and turn to a specified channel, a period of time may elapse between the receipt of the power on command until the television warms up and is ready to receive additional commands. Such initialization periods vary from appliance to appliance and therefore means is needed to allow a user to adjust the time delay as necessary.
There are various other problems with known systems which utilize voice recognition to control appliances. For example, one problem occurs as the list of voice commands grows. Generally, each time a speech recognition product detects a spoken word, it compares the word to those in a recognition vocabulary. In general, as a recognition vocabulary increases in size, recognition time increases and accuracy decreases. The more words in a recognition vocabulary, the more likely the speech recognition circuit will make a mistake and accuracy generally diminishes. Without the capability to utilize multiple vocabulary recognition sets, the commands available for selection must all be available at the same time, meaning they must all reside in the same recognition vocabulary set. Thus, the user may be inconvenienced by the limited command availability of a small recognition vocabulary or the diminishing speech recognition accuracy and longer recognition times resulting as the single recognition vocabulary is increased in size. Another disadvantage of a single recognition vocabulary is that the recognition of voice command always yields the same response. In a multiple recognition vocabulary set scheme, the same voice command may yield different results upon recognition if placed within different vocabulary recognition sets. A still further disadvantage of a single recognition vocabulary is that the recognition vocabulary words must be phonetically distinctive from each other which may limit the use of words that may be intuitive for their intended function but are too phonetically similar to other words in the recognition vocabulary, forcing the user to remember a word that is less intuitive. In a multiple recognition vocabulary set scheme, words that are not phonetically distinctive from each other may still be used by placing such words in different recognition vocabulary sets.
Another problem with known systems occurs when a device is limited to executing software for a single speech recognition mode for recognizing spoken commands to remotely control appliances. This problem becomes apparent if an appliance has the ability to process a series of remote control signals arriving within a predetermined time of each other differently than if the codes arrive at intervals outside of this predetermined time. For example, many televisions are capable of receiving the remote control signal transmissions for the digits one, two and three in rapid succession and responding by switching to channel 123 instead of changing to channel 1, then to channel 2 and then to channel 3. As such, a user must be able to quickly submit a voice command following the recognition of a prior voice command, and then the recognition of the latter voice command, the submission of a remote control signal and the detection of the signal by the appliance must occur within the time limit if the appliance is to interpret the commands as a related series of commands instead of individual commands.
Thus, there is a need for a stand-alone, wireless remote control that can be positioned in a location remote to both the user and the controlled appliances, which can be activated and operated entirely via voice commands, conveniently compatible with commercially available remote controlled appliances, and can be operated by a plurality of users and can be programmed by a user to operate in a customized manner.
The present invention relates to a method and apparatus that enables a user to control the function of one or more electric appliances or other electrical equipment solely with voice commands. An important aspect of the invention relates to voice-actuated mode switching for switching the present invention from a low power consumption mode.
A hands-free speech recognizing transmitter in accordance with the invention, which may be a light-weight small base mounted unit, is adapted to be located near the controlled appliances. A voice-operated selection mechanism is provided for selecting any one of a number of categories of appliances to be controlled. A universal remote control code library may be provided which allows codes to be selected for control of different types of appliances as well as appliances manufactured by different manufacturers. Infrared signal receiving, decoding and storage capabilities further enhance the invention by providing means to learn infrared codes from other remote control transmitters, such as an appliance""s original remote control transmitter. This feature is advantageous for appliances whose codes are not contained in the universal library stored in the invention""s memory and also serves to render the invention much less susceptible to obsolescence than speech recognizing transmitters disclosed in the prior art.
A scheme is provided where both speaker independent and speaker dependent vocabulary may be used as voice commands. A speaker independent vocabulary structure provides the user with the means of navigating multiple linked recognition vocabulary sets. At any one time, a single recognition set is made active by the speech recognition circuit. A voice command in one recognition vocabulary set may be used to activate another recognition vocabulary set. In this fashion, a user may verbally navigate among different types of remote control functions and target appliances in an intuitive fashion, while minimizing the number of alternative words from which the speech recognition circuit must select during any given recognition process.
With the total recognition vocabulary partitioned into smaller recognition vocabulary sets, improvements result in recognition time and accuracy because, in general, as a recognition vocabulary increases in size, recognition time increases and accuracy decreases. Furthermore, recognition vocabulary sets are created such that the contents within each set have either functional similarities or otherwise share a relationship that is intuitive to the user. To prevent confusion that may arise with respect to the active recognition vocabulary set at any given time, the present invention monitors conditions with a time-out counter and an error counter to automatically return to a general default mode if certain conditions are met. In other words, a predetermined number of recognition errors or a period during which no successful recognition occurs will result in a known default state.
Custom speaker dependent voice commands may be trained by the user and associated with infrared signals during product setup processes. These speaker dependent commands may be accessible from the vocabulary structure described above.
The present invention has the capability of transmitting multiple control signals, such as infrared signals, to one or more appliances in response to a single voice command by the user. Such macro programs may be programmed by the user during a product setup process. For example, a user may desire that the response to the voice command xe2x80x9cVideoxe2x80x9d is for the invention to issue commands to turn a TV power on, wait for the TV to warm up until it is ready to receive control signals, such as infrared signals, set the channel to xe2x80x9c3,xe2x80x9d select the VCR for control, turn on the VCR, and execute the VCR""s play function. As another example, the user may desire that the response to the user-trained voice command xe2x80x9cSportsxe2x80x9d is for the invention to issue commands to a TV to turn to channel 123.
The system may be configured to be self-contained in a single free-standing or stand-alone housing. The housing may contain a directional microphone mounted to be rotationally adjusted by the user to enable the user to adjust the sensitivity of the microphone so that the microphone""s high sensitivity region is pointed toward the user and microphone noise generating sources, such as an appliance""s loudspeaker are located in a region of low microphone sensitivity. Visual indication lights may be provided in a manner to remain in line of sight with the user to communicate operation status, such as availability for recognition and results of prior recognition attempts. A small keypad may be provided to assist the user during product setup sequences. A speaker is also provided to enable output speech synthesized prompts to further assist the user during setup sequences.